Kernel Mean Estimation and Stein Effect
نویسندگان
چکیده
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is an important part of many algorithms ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given a finite sample, an empirical average is the standard estimate for the true kernel mean. We show that this estimator can be improved due to a well-known phenomenon in statistics called Stein’s phenomenon. After consideration, our theoretical analysis reveals the existence of a wide class of estimators that are better than the standard one. Focusing on a subset of this class, we propose efficient shrinkage estimators for the kernel mean. Empirical evaluations on several applications clearly demonstrate that the proposed estimators outperform the standard kernel mean estimator.
منابع مشابه
Supplementary Material to Kernel Mean Estimation and Stein Effect
Stein’s result has transformed common belief in statistical world that the maximum likelihood estimator, which is in common use for more than a century, is optimal. Charles Stein showed in 1955 that it is possible to uniformly improve the maximum likelihood estimator (MLE) for the Gaussian model in terms of total squared error risk when several parameters are estimated simultaneously from indep...
متن کاملData-driven Random Fourier Features using Stein Effect
Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang et al., 2014]. A limitation of the current approaches is that all the feature...
متن کاملKernel Mean Estimation via Spectral Filtering: Supplementary Material
This note contains supplementary materials to Kernel Mean Estimation via Spectral Filtering. 1 Proof of Theorem 1 (i) Since μ̌λ = μ̂ λ λ+1 = μ̂P λ+1 , we have ‖μ̌λ − μP‖ = ∥∥∥∥ μ̂P λ+ 1 − μP ∥∥∥∥ ≤ ∥∥∥∥ μ̂P λ+ 1 − μP λ+ 1 ∥∥∥∥+ ∥∥∥∥ μP λ+ 1 − μP ∥∥∥∥ ≤ ‖μ̂P − μP‖+ λ‖μP‖. From [1], we have that ‖μ̂P − μP‖ = OP(n) and therefore the result follows. (ii) Define ∆ := EP‖μ̂P − μP‖ = ∫ k(x,x) dP(x)−‖μP‖ 2 n . ...
متن کاملEstimation of the Multivariate Normal Mean under the Extended Reflected Normal Loss Function
متن کامل
THE COMPARISON OF TWO METHOD NONPARAMETRIC APPROACH ON SMALL AREA ESTIMATION (CASE: APPROACH WITH KERNEL METHODS AND LOCAL POLYNOMIAL REGRESSION)
Small Area estimation is a technique used to estimate parameters of subpopulations with small sample sizes. Small area estimation is needed in obtaining information on a small area, such as sub-district or village. Generally, in some cases, small area estimation uses parametric modeling. But in fact, a lot of models have no linear relationship between the small area average and the covariat...
متن کامل